- Cracked the engima machine in WWII
December 3-5, 2019
What do Alan Turing and Nate Silver have in common?
Method for figuring out unknowns that requires 3 things
When we know the parameters we can simulate data
Using Bayesian inference, we can estimate parameters from data
To understand it we need to know about
Probability is the chance of an event occurring, defined between 0 and 1.
Random Variable is the variable that takes values depending on outcomes of random events.
Probability distribution describes how the probabilities are distributed over the values of the random variable.
To define a probability need
The probability must be non-negative for each value of the random variable and the sum of probabilities must equal 1.
If two events (A and B) are mutually exclusive, the probability that either occurs is the sum of their individual probabilities \(P(A \cup B)\):
If two events are not mutually exclusive, we need to correct for double counting:
Bayes’ Rule is a recipe for turning around a conditional probability so we can learn about the thing we are interested in.
Use Bayes’ Rule to go from the likelihood \(P(\text{Data}|\text{Hypothesis})\) to the information we want:
Bayes’ Theorem:
Probability of Testing Positive: \(P(\text{Test Positive})\)
2 groups will test positive: Infected and Uninfected
\(= P(\text{Test Positive} \cap \text{Infected}) + P(\text{Test Positive} \cap \text{Uninfected})\) which is equivalent to
\(= P(\text{Test positive}|\text{Infected})P(\text{Infected})\) + \(P(\text{Test positive|Uninfected})P(\text{Uninfected})\)
Being infected and uninfected are mutually exclusive so
\(P(\text{Uninfected}) = 1-P(\text{Infected})\)
Probability of Testing Positive: swapping in what we know
\(= {\color{blue}{P(\text{Test positive}|\text{Infected})}}{\color{red}{P(\text{Infected})}}\) + \({\color{orange}{P(\text{Test positive}|\text{Uninfected})}}(1 - {\color{red}{P(\text{Infected})}})\)
Gives us: \(= {\color{blue}{1}}\times{\color{red}{10^{-6}}} + {\color{orange}{10^{-2}}}\times(1 - {\color{red}{10^{-6}}})\)
\(=10^{-6} + 10^{-2}+ 10^{-8}\)
\({\color{green}{\approx 10^{-2}}}\)
\(P(\text{Infected}|\text{Test positive}) = \frac{{\color{blue}{P(\text{Test positive|Infected})}}{\color{red}{P(\text{Infected}}})}{{\color{green}{P(\text{Test positive})}}}\)
If we plug in what we know:
\(\frac{{\color{blue}1}\times{\color{red}{10^{-6}}}}{{\color{green}{10^{-2}}}} = 0.00001\)
Phew! Even though false positives are unlikely, the chance you are a zombie is roughly 1 in 10,000.
Bayes’ Rule is general, our hypothesis and data can be any events. We are often using it in our models to assess our parameters.
\[ P(Parameters \mid Data) = \frac{P(Data \mid Parameters) \, P(Parameters)}{P(Data)} \] - Prior Knowledge + Current Data = Current Knowledge
Prior distribution: what you know about the parameters, before getting the data - \(P(Parameters)\).
Likelihood: based on modeling assumptions, how likely are the data if the chosen values for our parameters are correct - \(P(Data|Parameters)\)
Posterior: probability of the chosen parameters given the Data, \(P(Parameters|Data)\).
We can keep adding data and updating the posterior (our current knowledge of the process) as data becomes available.
Draw a prior for the percentage of the earth that is water.
Who would win in a fight between Thomas Bayes and Ronald Fisher?
Can incorporate prior observations and prior knowledge
Useful in complex models with missing data and several layers of variability
Can be used to make management decisions based on our data, including evaluating unlikely scenarios.